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® VIDEO/AUDIO MULTIPLEX TRANSMISSION SYSTEM. 

2 bVanc^ Vid6 ° « «*> s,gna,s. The video and audio s.gna.s 

of a limited l^^^S^ZS^i of" the S^*"*"*"*" CaPabNity ,U " y und « r th ° Cond ' tIon 
detected. According to this detection Z *J£L> I ,nformat,on con «nts of the video and audio signals .s 

ratio of the video and audio IZ^JfVZZZT 0 " W COntr °" ed in res P ect 10 tne transmiss.on 

two signais. Hence, the^^ belween the P~-ino speeds of the 

insufficient transmission capacities * ,mpr ° ved ' 8s Pecally in TV conference systems having only 
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VIDEO AND AUDIO MULTIPLEX TRANSMISSION SYSTEM 

TECHNICAL FIELD 

system. It is becoming important to performing the coding and decoding of the two ai a g 
to maintain a balance between the audio data and video data. 

BACKGROUND ART 

t, a video and audio coding syatom Known in m. P«* ««* *mmm co*d ^-JJ-W-- 
„«h video oodod data and o*er controi data. >" »» case ,28 La lino. 

, rate such as in a 2 x B system which uses <™«Jf ™ ^SIT SaSto and video signal 

or the B system which uses one 64 kb/s line, the ^^TT^J^^Z is 56 kbrs and the 
,s in genera, nxed ,0 atouMt, £™£ ^ J^JS '£51 *«M *P~> » « «" 

o is 16 kb/s and the video coding speed ts 112 kb/s). inherently different 

However, the ratio of the data density between ^^^£^%Z^ in the case of a 
by several hundred-fold. Despite this, the data .s transmuted by the f™2 * m e ™ome S small, inviting 

so re:":.:.: rrrr a sys«.m. — - - — - - . — - « - 

the video quality and audio quality. ^ tl i« ril ovinn astern the redundancy of the 

„ a ^:?.^«es nT,^ » »cTe synchrony hetween majeure and sour* ».« ,e=.,v,ng 
M ,«e~ and an unn^ra, ^j*^ - J- - P „ p „ „ 

be sufficiently improved. d ta so not a || Q f the 

Norma.ly. also, the amount of the video data is far great ^ video data 
video data is actuai.y transmitted. That is. the portion which , no tran^d .^ignored ^an 
is sent with lapses. Due to this, the picture reproduced at the receiver s.de appears y 
4S This is because picture quality is less important than real time ^missior ^ 16 
On the other hand, with respect to audio signals there are silent jawrtr .nw-* P 
kps/s worth or 56 kps/s worth of transmission capacity is secured. Therefore, it is desirao.e 
periods in audio signals for the video signals. 



so 



DISCLOSURE OF THE INVENTION 

soTto maintain the optimal video quality and audio quality of the system as a whole. 
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rsJ?"J72£ reali .r? 38 fo " ows according to one aspect ° f the " resent in ^«° n . 

with J , P transmission s V stem " hi <* is comprised of a transmission unit provided 
sZT^cZZZ f aUdi ° inPUt t0 * gitai S0Und ' an 3Udio codi "9 tor "-"B Coital 

nuZLnn £Z ™? 1? ^ SOUnd ° f 3 f0nTiat 9n3b,in9 se,ection of 1,19 of transmission and 

un*T!c£nT,TT<? ta : 30 A/ ° COnV9rt9r f0f C ° nVertin 9 Vid9 ° input t0 a » "deo "ding 

de lm ninr a 9 t«nl? ^ 3001 ° UtPUttin9 th9 r8SU,t 38 COded vid8 ° date - a «*, control unit 
of Z c? a le«i ^TST 0 V aid C ° d9d S ° Und a0d S3id C0d8d pictur8 in accordance with the amount 

Ication slLTL T . , Sa,d COd6d S ° Und and S3id C ° ded piCtUre and 018 ' 8Sult « a " 

1h a T, a « ■ , m " lfap,exm 9 un,t for multiplexing said coded sound and said coded picture based on 

S12^|31JS ^ ^ inC ' Udin9 ^ a " 0Cati0n Si9na ' 50 as t0 obtain a certain 

SESTJirJr^ ^ .? reC8Pti ° n ° nit Pr ° Vided With 3 demul «P'exing unit for receiving a 
STS^TlSlJr-,*? fr3ns ™ sion Unit t0 a Mission line and sent through the transmission 

Z^ LTZotZ 9 V° V*? S ° Und ' C ° ded PiCtUr9> and contro ' data inc,udi "9 « ■fce-on 
nloint f f ! 1 9 * ° r deCOdmg said coded sound to obtain a decoded digital sound, a video 
for nZ^l ?T t 9 S3,d C ° ded PiCtUfe t0 ° btain 3 d8C0ded ^ a ^coding contro. un* 

SLST^ST 8 1 ° r 33 aUd '° d8COdin9 Unit 3nd Said video decodina unit baS8 d on said allocation 
signal a 0/A converter for converting said decoded digital sound to an audio signal, and a D/A converter for 
convening sa,d decoded digital picture to a video signal, the transmission being 9 periled by £S£ the 
transmission ratio of the sound and picture in accordance with the content of the Emission 

Therefore th« ZtZIZ ? ^ Vide ° bUt delations thereof etc. are not allowed. 

Therefore the transmission ratio of the picture and sound was fixed and audio data of a certain quality was 
sent regardless of the content of the transmission. q v 

In the present invention, the transmission ratio can be changed. Of course for this it is neeessan, to 
change the transmission ratio without interruption of the audio data. t£S£ J££ IpT^Z s 

2 oual *2 f2 k*: 3 ? ' S Se ' eCted in accordance the content of the transmission, so the 
ZX Tl ll eTnT' "I*" 9 S ° Und ' S neV9r interrupted ■* fra "^ission can be performed at the 

Sirr^r state at 1,131 time - nep — - p9rf °™ d * 

30 « h 2^^^«2S hg ^ r3teS ^ C ° mpared and *• ™ St *-r. ■« - bit rate. ,s 

ran.™*- 3d3PtiV ! T ° 0der ' S 3,1 SB - ADPCM - Th * Performs the coding dividing the data into a low 

SSeTJ! IT , d3t3 d6nSity 3nd 3 N9h re9 ' 0n freqU9nCy Pa ' with a ' ow data density 
selection of the number of bits only causes a change in the sound quality 

35 m '"k" 1 , 3 C3 !f V thiS SB " ADPCM and a Plurality of coding bit rates, the system as a whole can be made 
vTo 2?2 1H 7 ,eCt ? h thenumber °< bi * Q^O consideration not on.y to the audio data X. IZ 
among "Snes ^ ^ ^ AS SUCh Vid9 ° d3t3 ' USe is mad8 <* ln8 ' a < 8 »f change of data 

40 data^efore^f ?s° SL5^?^ 5 P9rf ° rm Va " 0US tVP9S °' Pr ° C9SSin9 f0r compressing the video 
aata. Therefore, it is comprised of a video coding unit for coding and quantifyina the diaitai oicture a 
vanable length coding unit for coding in variable lengths the coded video data a buffer in wni?h Ts is 

S ZZ^Z° T Pt6XinQ - 3nd 3 bUf,8r jUd9ement unit for showi "9 ^e state o storage of h 
Whin t 9 °r tPUtS 3 St ° ra " am ° unt S,9nal showin 9 tne state of buffer storage. 

To mUe the vWaoand 65 h ' ^ ^ Si9n3 ' " d9Cimat9d 3 " d time is taken until transmfssion. 
To make the v.deo and aud.o output match without deviation regardless of the content of transmission 

^n ^Tcote^T T° astern comprising a "ansmfssion 

I „ , . J converting audio input to digital sound, an audio coding unit for coding said digital 

generating de^av'ZTata fo^*? 9 Vide ° inpUt l ° digita ' PiCtUre " 3 delay amount calculation un^fo 
fnnnt Z 9 ™ t > T. f ° synchronizing video reproduced output and audio reproduced output from 

data an C 2" t °l T ^° ^ ^ a " d 3 multi P ,exin 9 «* mu.tiplexing the coded video 

delSexS a fn 3 '/;" 3 d3t3 ' 3nd 3 reC9Pti ° n unit prav,d9d with a den?u.tip.exin g unit for 
SETS!?. a H m i U ' ,,plexed s, 9 nal from a transmission line into audio data, video data, and delay time 
date, a variable delay control unit for delaying the audio data in accordance with the delay time data an 

^nSt^^" 9 T° ^ fr ° m th9 Variab ' 9 d9 ' 3 ^ control unra DM converter fo 

^ZTL a nTca^JX t0 30 3Udi ° S ' 9na '' 3 Vide ° deCOdina unit tor decoding the coded 

picture, and a D/A converter for converting the decoded digital picture to a video signal 

that Is the siienfo^r^ V* ^'"^ ^ * S ° Und durin9 the same less *™ a certain level, 
that ,s. the silent penod. and transm.t the audio portion all together, there is provided a v.deo and audio 
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IS BRIEF DESCRIPTION OF THE DRAWINGS 

Rgure 1 shows a conventional example of a video and audio SyStem: 
Fig 2 shows the transmission format based on CCITT draft recommendation Y.221. 
Fiq 3 shows a conventional example provided with a delay unit; 

„ % 4 , » *. c ^ ^■iST.SSSX "i. 

Fig. 5 shows the transmission frame format in the case ot a ainerem. <unuu 
system shown in Fig. 3; 

Fig. 6 is a view of the basic constitution of the present invention. 
Fig. 7 is a block diagram of the transmitter side of an embodiment; 
25 Fig. 8 shows the audio frame format: 

Fia 9 shows the audio frame format of different bit numbers: 

Fta 10 shows an example of the constitution of the multiplexing and demu.tip.ex.ng unft 

Fig*. 1 1 shows examples of allocation of BAS codes and a table ol [codes; 

Fia 12 is an explanatory view of the demultiplexing of a coding brt rate from BAS data. 

data: 

Fig. 18 shows another example of a transmission frame format: 

Fia 19 shows a method of writing into the format of Fig. 18; , . . ifcs _ f thft 

Rg. 20 to Rg. 23 show exempli of the transmission frames when changing the number of bits of the 

^ d Rg a 2i shows an example of shifting of the selection unit of the bit rate to the multiplexing unit in the 

^FTg-Tshows^ example of the constitution of the muitiplexing and demultiplexing unit of the system 

40 shown in Rg. 24; . . . fiirQ . 

Pin 26 shows the changes in the transmission ratio of sound and picture, 

Rg. 27 shZs The concept of the transmission of data in the case of a change in the transm.ss.on ratio 

of sound and picture; 

Rg 28 shows the basic constitution of another embodiment; 
« Ro 29 is a block diagram of the transmitter side of the system shown .n F.g. 28; ^ ann(t of 

Rg! 30 is a flow cha? showing the steps for determining the bit rate by the interframe rate of change of 
the picture and the optimal audio bit rate; 

Fia. 31 shows the basic constitution of another embodiment: 

Rg. 32 is an explanatory view of the buffer judgement portion of the system shown .n Rg. 31. 
so Rg. 33 shows an example of a bit array of FAS data: 

Rg. 34 shows an example of the allocation of BAS codes: 

Rq 35 is a view of the frame format showing an example of the allocation b.ts of sound. 
Rg. % is a view for explaining the decoding operation based on the allocation bits of the sound. 
Rg. 37 is a view showing the in-channel connection protocol; 
55 Rg. 38 is a view showing an example of additional bits for inqu.nng about capacty .n the AC data. 
Rg 39 shows the basic constitution of another embodiment: 
Rg' 40 shows the concept of the frame data used in the system shown in Rg. 39: 
Rg. 41 is a block diagram showing an embodiment of the delay amount calculation un,t of the system 
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shown in Fig. 39; 

system' s'hown VS?***" " ° f the ™ ,tipl9xin 9 and demultiplexing unit of the 

Rg. 43 is a view showing the control data in the frame format of the system shown in Fig. 39; 
s Rg. 44 shows the basic constitution of another embodiment: 

Fig. 45 is an explanatory view of the transmission frame format of the system shown in Fio 44- 
Rg. 48 is an explanatory view of the processing in the time division coding unit- and ' 
Rg. 47 is an explanatory view of the feedback control. " ' 
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Before explaining the embodiments of the present invention, an explanation will be made of the 
conventional aud.o and video multiplex transmission system using Rg. 1 to Rg. S so as to facilitate 
fs understanding of the present invention. 9 facilitate 

is a N b^r a Iinr»m h r J* "XT' ^ re,erenC9 " Umera,S " to the s ™ ob i e «s. Figure 1 
ivstem^d sfSwTontrj J 6 COnStitUt,0n 0f a c °-e"«onat video and audio multiplex transmission 
system and shows only the transm.ss.on un.t and reception unit of the two terminal offices. As shown in the 
,„ ^ ra ' the a ° mput is carted to digital sound by an A/0 converter 1. then is coded by an audio 

ThV T m6Xin9 Unit 6 - ^ Vid9 ° input is Converted to a d '9 ital by an A/D 

coZ^di h^, V !" V ' d8 ° COd ' n9 Unit 4 - tnen is coded in variabte le "9ths. The variable length 

Ti™^^ h . '"T*, ^ ina buffer «* in P ut to tne multiplexing unit 6. The video coding unVt 

« JSZi JT!T? 3 « ^ COd,d S ° Und 3nd COded picture input to tne "^plexing unit 6 
are multiplexed at a certain ratio so as to give the transmission frame format shown in Rg 2 then is 
■ multiplexed with FAS. BAS. AC. and other control data and sent to a transmission line To 

code?^nd ,P JSS oT 1 ' r0n L the transmission ,ine 10 is demultiplexed at a demultiplexing unit 11 into 
hv » f.^STi h P'«ure and control data. The demu.tiplexed coded sound is converted to digital sound 

oL^t On ^0^ hi" A* 9 ; " ^ C0nVert9d ,nt ° a " aUdi ° Si9nal by 3 °' A conv ° rt ° r 1 3 ^d then 
o dSodino uT^ tZn • d ! mu,tiptexed coded P icture * converted to a digital picture by a video 
?2L TL J T * converted into a video signal by a D/A converter 15 and output. The audio coding 

Sl nSSJJl i! 9 12 ° f corres P° ndi "9 l VP« to enable a coded sound to be decoded 
to itsongmal form. The same applies to the video coding unit 4 and video decoding unit 14 

The case where a delay is anticipated and a fixed delay is given is explained below- 
™ Jim U, L 3 Sh ° WS COnve^,io^a, constitution of a terminal office in a video and audio transm.ssion 
ZUT 'u25 w 9 numer l 20 in fi 9 ure is a terminal office. 3 is an A'D converter for the video signai. 
41 is a video cod.ng unit. 42 ,s a variable length coding unit. 1 is an A/D converter for an audio signal 2 is 

Zn^ltJZ ^ U T J ' S 3 dS,ay COntr0 ' Unit 6 a " d 11 are mu » i P«exing and demultiplexing units 68 is a 

STsTs a ,o 8 3 Vid6 ° deC ° din9 Uni< - 15 ' S 3 ° A converter - 12 is an a "P'° ^coding 

unit. 13 is a D/A converter, and 19 is-a system control unit. 

The video signal is processed by the video coding unit 3 and then processed for codinq in variable 

Sis cold 5 ^ T™™ 9 ** demu,tip,e -9 «*» 6 and P n. On the other hand the auSS 

!S' S , C tf 31 "? f Ud, ° COd,n9 unit 2 to the 4 kHz band 16 kbps or 7 kHz band 56 kbps. Provis.on ,s 
made of a delay control unit 7 for giving a delay corresponding «o ,he delay for coding the v.deo signal The 
signal ,s de,ayed at the said delay control unit 7 and supplied to the multiplexing andVemulS^nils 6 

unit^to^ ^ mU,tiP ' e * ed 3nd •« *• interface 

The signal received from the opposing terminal office is demultiplexed by the multiplexing and 
vZ ! P 'T5 ""I? 6 "* 11 int ° 3 Vid6 ° Si9nal and audio S '9" a '. The video signal passes Tough the 

14 r d °' A C ° nVerter 15 ^ " f6,Ched 35 th9 video Th- aud^lgna" 

passes through the aud,o decoding un.t 12 and D/A converter 13 and is fetched as the audio output. 

Note thatttie amount of delay control T in the above-mentioned delay control unit 7 is given by: 

T - (t»| - t„2) * (t A l + t A2 ) 

where. 

ss l »< : delay time at the processing units 3. 41 , and 42 
tv2- - delay time at the processing units 1 4 and 1 5 
Ui: delay time at the processing units 1 . 2. and 7 
tAj: delay time at the processing units 12 and 13 
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Further, the transmission line has line speeds in multiples of 64 kbps from 64 kbp, Mo » £»t»d is 
fixed in capacity. Therefore, as shown in Rg. 4. feedback control is performed and the cod.ng processing 
the video coding unit 4 is controlled. Q . th f? aure 

• Rgure 4 shows the mode of the feedback control. Reference numerate 41. 4. and 19 .r .the figure 
correspond to Rg. 3. and 42 shows a variable length coding unit. 43 a buffer memory. 17 a first threshold 
value rS ?iS. second threshold value. In the buffer memory 43 is stored ^^J^£S^ 
date This is read out from the buffer memory 43 at a fixed speed and led to the multiplexing and 

^I'Si^ £ amount of video data stored in the buffer 

a restart of the video coding. Further, the quantification is made nner 
s Rgure 5 shows the transmission frame format in the conventional case. a ,| OC ated to the 

P,C X™ 5(8, show, m Name tenna, in ft. case of ,28 Kbps. Two cases exist kbps^ 
, aiiocafcd .0 !he sound (B-„ snd one when. =6 kbps is -joc-jd * « « 

a. ft. »ideo signal, as explained wdh relerence » ^4. ft. I * * 

«r:rd^— —on. .» - * 

30 transmission unit B and a reception unit C d audio mult ipiexing system of 

Rgure 7 shows one embodiment of the transmitter side ot tne v oeo ana ^ h codina 

the present invention, in which embodiment the audio coding unit 2 

units' (for exampie ADPCMs, 2, to 2, each with different ^^^^J^^ZTiJm 
sound, an evaluation unit 2A for determining a coding , W ^£^£T££ ^d generates a coding bit 

« S video data, and a buffer 43 for temporarily storing the variable length coded v,deo data. Further. 9 
a clock supply unit of the transmission frame of transmission and reception. 

The frame (packet) format of the audio data output from the adaptive coding units 2. to 2. is as shown 

dTsSs—^^^ 
45 optimal noise evaluation multiplied by the sampling rate n (n - 8 kHz). 

Examples of the 0 to 7 bit coded audio frames of the adaptive cod.ng units 2. to 2 8 are shown ng. a 
One of these is selected by the selection unit 28 and sent to the- 1 multiplexing unit embodiment . 

Rgure 10 shows an embodiment of the audio delay and multiplexing units 6 and 7 ^ em^bodj 
the un« is comprised of a CPU 21. a CPU bus 22. a ROM f^ 0 ^^^ e ^ o Z bus 22 
so the transmission line in the frame formats shown in Rg. 9. interfaces (l/F) 24 to 26 for e toning |io i 

vSeo data, audio data, and coding bit rate data, a RAM 27 having an address *^"%Z%* £ £ 
in the video data, audio data, and bit rate data from the interfaces 24 to 26. and an interface 
transmitting various types of data to the transmission line. ft 

The transmission frame format shown in Rg. 2 is one wh.ch allocates bits based on the 
55 recommendation Y.221. In the mu.tiframes. 16 transmission frames constitute one unrt one 
transmission frames), and 8 bits horizontally and 80 bits vert,*..* for to£ ^^S^^. 
transmission frame. Further, one transmission frame « ™npmad l of channel) 
alignment signal) data and BAS (bit allocation signal) data as the control data (AO lappucaoo 
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canl^'Z\?T r *T iS comprised 01 16 transmission frames, as mentioned above, but the data is 

tfT^^^V^'pT " 2 - 16 ' 5 data 30(1 BAS data ' however - are different 

'° the Sr^ILT eLmn? 9 ^ 3 2 am ° Unt ° f ^ inf0rmation and the amou "t °< video data at 
evlSo Ik thfcS ft" ? COd,n9 bit rateS) - By tHe reCeipt of tne codi "9 bit rate da * **» the 
nS^JZZw^^^ " ' h nC0,T,0rated in the BAS date - *own in Rg. 11 and transmitted: 

Next an explanation will be made of the transmission frame operation by the embodiment 

the iS^nSTlSL 8 " ^ C3Se 01 ^ ° f ab ° Ve ^ 8n fi ° ned Emission S data (Rg. 2>. 
» n thJ 1 r d 0n accordance w «h the coding bit rate stored as BAS data by the CPU 21 

1 BA? rar s a o y, m e e frarr\ COmPriSed ° f " 8 * bitS " e,iminatin 9 » e fra ™ header by indudin M in 
the BAS data so the frame allocation can be optimally performed as shown in Rg. 13. 

• L b rctTd^gr; a r e and - audio data is - s ™* 1 - ^ - sss^jk 

r^n 3 ^ 813 ^ VidB ° ^ ^ ,iNed " ' 0r each °"° sta 9« ° f vertical 8 bit units. The 
aSo daS. 9 POrt, ° nS fr ° m th6 bUffer 43 after ^P'^on of filling in of the stages of 

) audio* ^TJ^TZl^ ,S f T ed UliS Way - R9Ure 14 10 R 9- 17 snow the relationship between 
X a^d^t cSng * ^ ^ «" CaSe * codi ^ c ° d ^ 3-bit 

rs^^^ ^:r uded in ,he bas dat3 ' so the audi ° codin9 

bits 1 1^^Z!^J B indUded in the BAS d3ta in tnis wa * the data can b. stored for every 8 
TtSi^U ^ advantage of no partial data being caused as in the later mentioned Rg. 23. 

BAS^ataTsLl m„ tr f nsm ' ss '°" frame in the case where the audio frame header is not included in the 
horLntl! k , „ 9 - 8 - th ' S Case ' tne header is ""Sidered 3-bit data of one vertical bit and 3 

The transmission frame comprised of the audio data and video data is stored in the RAM 27 Wh«n 

m U ,tjrg jt%rn:%ins^ - — - - 

nitiSrffiE? S he S, ^ B h 3 d3ta ' and demu,ti ^ d ^na, data is passed Sough 
aud^c^ - the audio data to £ 

deco^ln^ « - data from the decoding control un„ and 

codfii^^iT^TJSr used f in 019 present invemion is fixed ,n wt ien 9 th - s ° to make *° 

dock ra e ts nTrfo^in h ! ! / ranSm,SS, °/ 1 frame - tne 0 P^ation synchronized w,th the transm.ssion l.ne 
^T^TS^tlST" 0 ^ the D I eCeiver Sid9 the c,ocl < the clock supply unit 9 shown in 
in SI Iho 1 . 2 9h However - ,n RS 42 2 or INCU. the clock is supplied by a separate system 

^gZ^^tBZ'TJZ 9 ':' r" 6 ^ ° U,PU,S ° f Wunits^ to i^npu, 

rougn the selection un.t 2B to the multiplexing unit 6. but in the embodiment of Rg. 24. the outputs of the 
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adaptive coding units 2, to 2, are input to the multiplexing unit 6 as they are and the select data is also 

sent to the multiplexing unit 6. g ^ output audio 

The system construction of the multiplexing unit 6 in this case » shown in Rg. 25 .Tne 
data of the adaptive coding unite 2, to 2a are received through the iM aces .25 j to 25^ . One of the p 
s of these is selected by the coding bit rate data and stored « ^ Coding unit 2 suitably 
In the present invention, the redundancy is compressed by the adaptive audio , cooing u y 
changing the bit rate in accordance with the amount of data of the .nput sound, and the coded aud,o 
and the bit rate data are sent to the multiplexing unit 6. from m 

By this, the multiplexing unit 6 multiplexes the audio data, bit rate data, ana me V ' Q « 

the reduction in the number of bits. demultiDlexed to video data, audio data. 

" ™T?Sr JSlLng o. M amcum o. audio dat, and I. amount 0, video da. changes. IN. i. 

"Tu^^T™ I. muitipiexed MM. data. As mm , in the ^ t^enS^d^ 



2S shown in Rg. 28. 



Ftaur. 29 is a block diagram of the transmitter side in this embodiment. .__.„.. , „ 

confute. 2 .a comprised 0. eight adaptive coding unite (to. ^""^„ M »,£ * * 

each - — « -J- _ - - J. J-J » ™Jd™ 

FuSe^reshold vale Th is adjusted by the coding control I unit 5 in ac^rd^ce w* the 
the storage amount of the buffer 43. The selection and mult.plex.ng unit 6 "eludes a selection u 
receives as input in paral.e. the coded outputs of »e adaptive^ from 
40 audio data of the corresponding adaptive coding unit by the v.deo and aud.o allocation 

^ "Tf^ptC'ormat of the audio data output from the adaptive coding unit 2, to 2. is as shown ,„ 

45 optimal noise evaluation multiplied by the sampling rate n (n = 8 kHz). 

Samples of the audio frames of 0 to 7 bit coding by these adaptive cod.ng un.ts 2, to 2. are shown 
Rg. 9. One of these is selected by the selection unit 6a for the ™>*»m results of tne 

Roure 30 shows a control algorithm of a cod.ng control unit 5 tor receiving tne _ res 
common of the rate of change comparison unit 82 and issuing video and audio allocation bit rate data 

50 M " ,n Rg. 30. first, the results of the comparison of the comparison unit 82 '^T^X^ZX 
rate of change of the video data between frames is larger than a ^threshoW alueTh (step S£ For ^ 
when participants at a video conference stand up. the v.deo data changes « emen °°"**'™° Qn bit rate 
va,ue Tn is exceeded, priority is given to the video data regardless of the video and audio £em£ 
55 data Ml by. as shown in step 2. producing and outputting v.deo and audio all oca ion bit ate a 
55 ^g on.y 2 bits (16 Kb., (see Rg. 9(3), to me audio data -^ng 14 bits <i *^J°^° 9 
data Further, the frame composition of 2 bits for video and 14 bits for video - 16 
multiplexing operation of the multiplexing unit 6. 
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h,,«o 0 , n ^ e ° th ![ t lT C V h9 f9SU,tS ° f the com P aris °" at step SI. if the amount of data stored VBI in the 

S2Tfl?iSSni . V3JUe Th ' ^ ,h9 rat9 °' Change 0f video data fr ames is sma er 

than the threshold value Th. are checked (step 53). As a result, when the storage data amount VBI is so 
large tha ,t exceeds the threshed value Th', the control proceeds to step S2 in the same wT as above anS 
5 pnonty allocation is given to the video data. • 

r a t«^rJf!n JZ' , Vide ° ^ 3Udi0 a,,ocation bit rate corresponding to the content of the audio bit 
rate data ACI from the evaluation unit 2A is determined at the following steps S4 to S15 

52 21 wh!!T? "I? T °J b !!. rat9 d3ta ACI indiCat6S 2 - bit COdin9 at ste P S4 - *° c °"tr°l P^eeds to step 

ro audio » i JH^i h S 3 ; b "! C ° din9 ' 13 bitS 004 kb/S) are a,located for *• v ^eo data In this way. the 

ouwt S m ,2 f 90 J? rat8 ^ d6Cided °" ^ ,he audio ** video Nation bit rate data Ml is 
output to the multiplexing unit 6. 

audio hL^^Zh ° n9d embodiment " the transmission ratio was determined from the optimal bit rate for 
d««l„ ™ , k ! r ! t8 ° f Cha " 9e ° f Vide ° data ' and the stora 9° amou "t of the buffer, but the 
rs £ ZsTTJZr flTV? T tVPe ° f ^ "* fUfther USe may 09 made of otner data relating to 
El t ° data " f ° r 9Xample - the ir, traframe rate of change of video data and the high 

frequency ratio of the audio data. a 

ADPCM !T2 9mb0diment K Uses ™ SB-ADPCM (sub-band ADPCM, as the audio coding unit 2. The SB- 
- * f ♦ tyP k ? C 63 ^ by dividinQ " in, ° *"* fr^^^y bands and allocates higher order 
and ?Li o T kT 6 / (f ° r eX3mP,e ' 6 b " 5 8 bi,S) 10 3 ' 0W ,rec I uenc y P° rtlon a data density 
?SU2T ? k,° 3 Sma,,8r " Umber (2 bitS) tD ,he hf 9 h fre ^ enc y ba "d with a low data density 

1 i ,- 3 Sh ° Wn m pnnCip,e in Rg - 31 • * e transmitter side is provided with an A,D 

SET JZ C °Z 9 T 9 ^ aUdi ° inPUt l ° di9ita ' S ° Und - 3,1 S8 ' ADPCM codin g «* 2 for coding the 
SS^^t?^ 3 w freqUenCy POrti ° n ^ 3 hi9h freqUenCy bit POr « 0n - an " D con ^ 3 
1m Ti, ' nPUt t0 3 d ' 9 ' tal piCture - 3 video codin 9 unit 41 f °r "^"9 said digital picture, a 
Z^n t 9 «*"B «"* 42 tor coding in variable lengths the said coded video data, a buffer 43 for 

a^ 9 J.n, 0 nn a V t V3nab,e ten9th C ° ded Vide ° data ' and 3 bu,fer < ud 9*ment unit 44 Sr determining ii 

for SST £T 2 J"?'*! ^ 30 3m ° Unt ° f d3ta St0r9d in tne buffer and a multiplexing unit 6 
for al.ocat.ng a part o the v.deo data to the high frequency bit portion of the SB-ADPCM coding unit 2 in 
accordance with the allocation bit rate and multiplexing the allocation rate data, video data, and audio data 

s ian7TZ'^/ eCe,VW Si<i9 r ' - 3 ^multiplexing unit 11 for demultiplexing the multiplexed 

s.gnal from the transm.ss.on l.ne 10 into audio data, video data, and allocation data a D A converter 13 for 

vTdTdaTa 7 SB '^° M COdSd dig, ' ta ' S ° Und t0 3n 3Udi ° Sig " al - 3 buffer 143 f °r L£%Z?Z£ ^ 
I« ,n o h 8n9th d8COd,n9 UnH 142 f0r decodin 9 in variable le "9 tn = the video data of the buffer 

dSa TJdlT L°nT 9 nH 141 f ° r iWerSe C ° ding ° f the V3ri3b,e length decoded vi<J eo data and issuing a 
SSi„™, 9 C ° nVerter 15 f ° r convertin 9 tne digital video output to video output, and a 

TtoeZntZ h ° r ^ ntr °" ing the 3Udi ° d6COding Unit 12 in a ccordance with the allocaUon rate. 

f £ r.r I ° 9 ' 31 • th9 Vide ° inpUt iS COnv9rted to a di 9 ita < s'gnal by the AD converter 3. 
T^ouT^LT. *i t0 T COdin9 Unit Wher8 the d3ta 0f ,he pr,or frama is used tor com P ress.on 
*,f?J^l 1 w ^ Ue ° bt3,ned ,h8r8 the " being ""antifled. This quantified data is processed by 

occur^l and 9 ln J ""^ (VV ! LC> 42 S ° th3t ^ C ° deS " 3 " 0Cated for date «Bh frequencies of 
occurrence and long codes are allocated for data with low frequencies of occurrence. The coded data is 
stored temporarily in the buffer (BUF) 43. 

of thl^n!! 3 , !' the buf,8r 43 is multiplied and sent out by the multiplexing unit (MUX) 6. When the capacity 
of he ransm.ss.on ,ne is small, the amount of data stored in the buffer 43 increases and when the capacity 
of the transm.ssion line is large, the amount of data stored becomes smaller V 

the J^^'h ? T S>b J e t0 ° bt3in 3 9r3SP ° f the 3m0unt of data transmitted to the transmission line by 
buffe ^^udoelnf 3 n> I'' bUff8r ThiS d3t3 m3y be Sem t0 the buffer i ud 9«ment unit 44. and the 

so me^STdia det8rmme the 3 " 0Cati0n tooPir-g) rate of the audio data based on the data on 

The audio input is converted to digital sound by the A D converter 1 and sent to the SB-ADPCM coding 
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Therefore the allocation rate decided on at the buffer judgement unit 44 is used for selection of the 
to t^ cull the SB-ADPCM 2. the required number of audio bits are cut away, video data is allocated 

ISLS^SSm^T 3Udi0 data ' Vid8 ° d3ta ' 3nd al,0C3,i0n ra,e data are mu.fp.exed by the 
multiplexing unit (MUX) 6 and transmitted. 
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,n this way. the amount of audio data can be changed in accordance with changes in 
video data in the same way as shown in Fig. 26 and Rg. 27 and a more supenor quahty of video data can 

^ shows an embodiment of the buffer judgement unit 44 ^^'^S^S ' 

receives as input the writing address W to the buffer 43 compnsed of a RAM from 

coding unit 42 and the reading address R to the buffer 43 from the multiplexmg un,t 6 and performs the 

^WHenT- 9 ^ threshold vaiue Th,. since there is no progress in the reading of data J* ~J ' 
Z writtng of data to the buffer 43. it is judged that the amount of video data generate «.s large, ajddato 
m i) is sent to the muitiplexing unit 6 reiating to the allocation rate 

lo the high frequency component in the 8 bit audio coding data from the SB-ADPCM cod.ng un,t 2 and 

for cutting away the 1 bit in the 2 bits ailocated to the high frequency component -n the 8 brts oflauao 
cod^g data from the SB-ADPCM coding unit 2 and coding only the remam.ng b,t and the 6 brts allocated 

the 8 bit audio coding data from the f^^Tlol to Rg 33 The frame synchronization (1) is 
An example of the bit arrangement of FAS data .s shown in F.g. 3J. me ^ y muttjfr ame 
performed by recognition of the FAW (frame alignment word) (m Rg. 33. 0011011 ). m ™ 

° f " 0 Rgu 0 re 1 34 shows an examp.e of the BAS data. This BAS data is produced bY ^ng the rati o be^een 
the Sount of the audio data and the amount of the video data ^^J^%?%?£ £ 
transmitter side (see Rg. 12). It is data used for *e demu.tp.ex.ng of < ^^^^jTrnMrmm 
frame synchronization. The unrts of change of ^^^^ ^ ^ ^om^e. 

m ^TJSX^ operSn. the multiplexing unit 6 using the above frame format de^s ^ bit 
arrangement using the BAS data based on the a.location rate data output from the codmg c ^ro. unrt 5 as 
arrangemem us na Q bjts Qf gudl0 data as ^gy are ((a) 

TZZ me cts > 7Z»Z 1 bS ^ng 7 bits ((b) in the figure), and the case of discarding , 2 brts 
anSg onlylne 6 bits of the'.ow frequency component ((c) in the figure) and stores the aud,o data and 

^^eTthe^taTsem to the transmission line at a rate matching the transmission speed at the point o. 

" m *^:™l^^l^X« - vaHous bit .locations per frame of the audio data 

in Rg. 35 are as follows: 

(a) 6 x 80 (480 bits) x 100 frames (1 second) = 48 kb/s 

(b) 7 x 80 (560 bits) x 100 frames (1 second) = 56 kb/s 

(c) 8 x 80 (640 bits) x 100 frames (1 second) = 64 kb/s ^nctniption as the 
On me other hand, the demultiplexing unit 11 at the receiver side has the same constructs 

multiplexing unit 6 shown in Rg. 10 except the arrows are in the reverse direction ast _ b|ish _ d for jnput 

That is (D Y.221 frame synchronization and (2) multi-frame synchron.zat.on are establish ed i tor mp 
data^om the transmission line, as mentioned before, by analyzing the FAS data Then the data .s 
d muSxed by the demultiplexing unit 11 based on ^ ^ £ 

through the interfaces, with the audio data going to the SB-ADPCM decod.ng un.t 12 and tne 
beino sent to the buffer 143 and the variable length decoding unit 142. 

Sere the demultiplexed audio data must output the data in accordance wrth the number of aud-o 
allocation bits as shown in Rg. 36 at a timing synchronized with the transrn.ss.on l.ne clock (8 kHz. 64 kHz). 
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is deooded from ttte buffer 1*3 bTIV^Lu^I'^ - C0nV,r,Or t3 ' Fur ""'- »» "'**> 

adaptive audio codinri unto r, . ™? «• DPCM syslem, APC-A6 system, and other systems of 

and^ceptio^ 

•o med^e may £ti 8 7£ TcpT^ ""T"" ° f ^P^ 568 find a com ™" •P™*n- 
wherein 9 ' CP (,n - channel connection protocol) procedure as shown in Rg. 37, 

A iS I 0 " 1 !!! ° n ' the " the FAS data is used for fra ™ synchronization. 

ft h~o" A C 9 data. ^ CaPadtieS ' ^ * - ~ are inguired about by 8 bits 

" (Ho wTver'Tnthe^^noT" ***** °" ^ * *" is des * nated usi "9 *. BAS data 

co^ng r^e , " SyStem ' ta exeCUted for communication using a fixS 

conJenron\,TpVats matiC * * h P0SSib,e to ™« competition with the 

coding and decoding d.fference between the processing speeds of audio and video 

e. The sziras ps^ : n y r irrr fe of a tra r ssion unit ° - a »— » - 

^ng^c^ * *" d a ™~ - - * 

signalTm' Z ™sZ«£ L'Toll T« * T??** "* " * de ™<^ the mu.tip.exed 
control unit 32 to^ES H ° aUd '° data - v,deo data ' «* de,a V «m. data, a variable delay 
12 for decoding the ^ m» !" accordance witn «• delay time data, an audio decoding unrt 

variable amount) reouired "lor L ™h 9 9 aUd '° S,9 " al is shorter tnan d e»ay t.me (Tv: 

Ther^: 6 " I ; 8 ,H de,ay ^ Td ^ be f ° Und by findi "9 Tv ' s,n « > T * * constant 

. multiplexing unit 6 date aTng to the th f VK ?° C ° ding unrt 4 9iv6S to the 

known constant delay time ?a Th s delafZ 2? -J J , ? *" **** ,ime Tv and the Piously 
audio data (see Fio 40) V * '* mu,,, P ,exed and transmitted with the video data and 
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55 



cateolatioo cor^o^^^^fS^^™.* *»■ «*» » ™-='V 35. and a 
soieo, si 0 nei. and reoeivin, an 0,^^^^^^^ "° " 9 " a,Ua "° n ^ 
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™ Zetan 4 ."e^^< vau» (n, ^ on ,e - « «*« 

of mo video data amount dotocdon unit 2c. lor anmf*. _ cateu |«|on oontrol unit 2d. 

' „ r^^r^ro daH S-tTS. - - J— - ^ 

when the evaluation pattern is input at the coding unit 4. it is output in me ur P 
time Td for obtaining synchronization. This is found from 

20 r-r-t: ; x - JaCr r. s ^ » oo*„ M •« M ™ «« - » - 

the multiplexing unit 6. . embodiment it is comprised of a 

Rgure 42 shows an embodiment of the ™** to ™* U ^^ S ™ ™ Z Jansmission line in the 
CPU 21. a CPU bus 22. a ROM 23 storing programs to r 1 mmM "J*"^ TjJZmt data, audio data, 
frame format shown in Fig. 2. interfaces (VP) 24 to *£*^*XJ££ Taving^ddress space for 

which is used for establishing synchronization . by Mh. ^^^^rgufshed by the above 

amount of audio data and the amount of video^ata^ ^Z ^lZ^^Ts^m^e (1 
as synchronization for demultiplexing of the data. This BAS data is juage 

multiframe = 2 sub-multiframes). indicates the data relating to the 

Further, the delay time data from the deiay ^^^^.^SSZ^i unit) for the contro. 
delay time Td. mentioned above, by the use of 8 ^ of OLY0 to 7 C ODE ^ ^ ^ ^ ^ ^ 

data (AC data) unit shown in Fig. 43. These 8 bits enaoie 
40 receiver side apparatus. f f the a bove frame format. 

Next, expiaining the operation, the muKip.ex.ng unit 6. in ^ ^J.^^ in ^ C PU 21. then 
decides on the bit arrangement in ^^^J^^^Z Slay time data found by the 
stores the audio data and video data m the ™ ™ JJ*. data £ sent to the transmission 

delay amount emulation unit 31 is stored as ^^^fJ^Z ir^e format is completed. 
« line 10 at a rate matching the transmission speed at *e same construction as the 

On the other hand, the demultiplexing unit 11 at the receiver sicen?5 
multiplexing unit 6 shown in Rg. 42 but with ^J^^^^^^ are established for the 
That is. (D Y.221 frame synchronization and (2) J*""™ FAS data. Then, the data is 

input data from the transmission line. * data are sent 

so demultiplexed by the demultip.ex.ng un. 11 based on BAS date ^™ 32 ^ video data 

^r^d^ * - — data * simi,ar,y 

11. deiaysle audio d'ata by the delay time Td. and sends «J^*££Z^£» of deiay tables 



in advance, as — , , . „ T 

video data generated and use it as the video coding delay time T^. 
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<Example of Table> 



70 



250 msec 



200 msec 



150 msec 



Coding in frame: t vc - 500 msec 
Coding between frames 

(1) Large amount of data generated: T, 



(2) 



(3) 



Large amount of data generated: T v 
Large amount of data generated: T 
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an,.. * pon,on <* "1 <ransmn lor a Sme corresponding to tne delay 

Sl^Z , 9 d ' V,S,0n COd,ng unit - 3,1(3 72 a time <""W°n decoding unit 

examote aSLT^L ■ PP t0 ,he mu,ti P ,exin 9 and demultiplexing units 6 and 11 in for 

STt? notifies' he system TZnl" ? ?o° ^ ° f 7 U " itS < 56 kb ^ Ne * th ° "me division codmg 

^J^^Z^S^* SyStem COntr °' Unit 19 chan 9* s «• threshold va,ue (second threshed 
buffer ! SSS2 ,n ? 4 ; d T e h ° C ° din9 Unit 41 based « *• amount of data in the 

^ r "~ " - video si9nal 

o sg ai and desenbe ,n the frame data the data for notifying the opposing term.nal station of the 
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transmission speed of the audio signal in the of tne embodiment and Fig. 45(B) 

Figure 45(A) shows the transmission frame format m the case ot me emuo 

explains the component frame data. header and has component 

described in the form of the data "000" to -111". min imum 0 kbps to 56 kbps is 

in the case of the present invention, one of the V°*° n *^™*l™TZ "variable" in fig. 45(A). 

allocated and sent as the portion for transmits ^ .^^J^^JSn audio signal, the 

corresponding to the above-mentioned audio transm.ss»n j pee* H ml kbps .s g.ven 

remaining illustrated (56 - a) kbps is used for transmittmg toe 

Figure 46 shows the mode of processing in the ^SSSCSTto converter 1 shown in Fig. 44 
WL an audio input© is given as iHustrated. t %J?J?£L during the effective 

, and becomes the A/D output ©. Note that 1 . 2 . ana a n_ me g Ambled as the illustrated 

period. The A/D output© has silent periods cut away as -llustrated and is assemo.ea 

packet output ®. n a ta (3> is oroduced like the illustrated (A). 

The packet output ®. as shown by the ..luxated jacket data ®. s Produce^ ^ ^ 

(B). (Q... for each predeterm.ned tome y. ™ 1S d f* ™£ } . , led to the multiplexing and 
o corresponding to the above-mentioned delay confrc I. amount T and « W*- ^ to ^ 

demultiplexing units 6 and 11 as 0 to 7 bits using 8 kbps as "J-^JJ speed is given depending 
multiplexing and demultiplexing units The ^^^° n ^ U f 8 ^nTcSe of J bit the "000" shown in 

iwxrAtt nsirr^" - - - case - r « —3 

!S Pr0d Rg e u d re 47 shows the mode* feedback, ^^^S^ 4 TS^« coding speed) is 
The audio transmission speed produced .n the time division cooing un i v 2 . jn me 

supplied to the system contro. unit 19 as the TocedTa^e in the same way 

case of the present invention. Note that the first threshold value is given oy a 

30 as in the prior art ^nmmandina the video coding unit 41 to stop the 

The system contro, unit 19 changes «. «nd*ons for ^^aSTSSSSn speed (coding speed), 
coding processing, as shown ,n Fig. 47 8). based ° n ™ ™° msux(is the above-mentioned stopping in 
Alternatively, it changes the table of quanbficaton. ™ * SSTS data *ored in the buffer memory 
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CAPABILITY OF EXPLOITATION IN INDUSTRY 

The present invention is used in video conferences etc. using ^^^^i^SSSZ 

r s ^m=f^ 



TABLE OF REFERENCE SYMBOLS AND MATTERS 

1 A/O CONVERTER 

2 VOICE CODING PART 

3 A/D CONVERTER 

4 VIDEO CODING PART 

5 CODING CONTROL PART 

6 MULTIPLEXING PART 

7 DELAY CONTROL PART 
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8 


CHANGING RATE DISCRIMINATING P 




9 


CLOCK SUPPLYING PART 




10 


TRANSMISSION LINE 




11 


DEMULTIPLEXING PART 


5 


12 


VOICE DECODING PART 




13 


D/A CONVERTER 




14 


VIDEO DECODING PART 




15 


D/A CONVERTER 




16 


DECODING CONTROL PART 


10 


17 


FIRST THRESHOLD VALUE 




18 


SECOND THRESHOLD VALUE 




19 


SYSTEM CONTROL PART 




20 


TERMINAL OFFICE 




21 


CPU 


15 


22 


CPU BUS 




23 


ROM 




24 - 26 


l/F 




27 


RAM 




28 


l/F 


20 


31 


DELAY AMOUNT CALCULATING PART 




32 


VARIABLE DELAY CONTROL PART 




41 


CODING PART 




42 


VARIABLE LENGTH CODING PART 




43 


BUFFER 


25 


44 


BUFFER DISCRIMINATING PART 




68 


TRANSMISSION LINE INTERFACE 




71 


TIME DIVISION CODING PART 




72 


TIME DIVISION DECODING PART 




81 


FRAME MEMORY 


30 


82 


CHANGING RATE COMPARING PART 




2A 


EVALUATION PART 




2B 


SELECTION PART 




A 


TERMINAL OFFICE 




B 


TRANSMITTING PART 


3S 


C 


RECEIVING PART 




D 


TRANSMITTING PART 




£ 


RECEIVING PART 
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A video and audio multiplex transmission system having a transmission unit (B) comprising- 
an A/D converter (1) for converting audio input to digital sound 

Z^SStZS* C ° din ? diQita ' S ° Und 3nd ° M,pUttin9 lX as coded sound of a 
enabling selection of the amount of transmission and outputting audio content data 

an AD converter (3) for converting video input to a digital picture 

a video coding unit (4) for coding said digital picture and outputting the result as coded video data 
nir h « 9 COntr °' un,t (5) for determining a transmission ratio of said coded sound and said coded 
SET ,n ti aCCOrd f nce w,th the amount ° f data of at least one of the sa,d coded sound and sa,d 
coded p.cture and outputting the result as an allocation signal, and 

!i£2„ PleXinS T l f ° r mu,tiplexin 9 said cod «d sound and said coded picture based on said 

trZTZ T ^ Urth6r COntrol daU indudin9 said al,ocation s '9" al s ° « to ^tain a certam 
transmission frame length. 

A video and audio multiplex transmission system having a reception unit (C) comprising: 

L d ^S eX,n , 9 . Unit f ° r r8Ceiv,n9 damu '«P'exmg a coded sound, coded picture, and 

control data including an allocation signal from a transmission line (10) 

an aud.o decoding unit (12) for decoding said coded sound to obtain a decoded digital sound 
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a video decoding unit (14) for decoding said coded picture to obtain a y*"^*?™ video 
a decoding control unit (16) for performing control for said aud.o decod.ng unit (12) and said v.deo 
decoding unit (14) based on said allocation signal, 

a O/A converter (1 3) for converting said decoded digital sound to an aud.o signal, and 
a D/A converter (15) for converting said decoded digital picture to a video s.gnai. 

A video and audio multiplex transmission system having a transmission unit (B) comprising: 

an A/D converter (1) for converting audio input to digital sound. «„_,.♦ 
Z audio coding unit (2) for coding said digital sound and outputting it as coded sound of a format 
enabling selection of the amount of transmission and outputting audio content data, 
an A/D converter (3) for converting video input to a digital picture. 

Tviteo coding unit (4) for coding said digital picture and outputting the result as coMvite data, 
a cotfng confrol unt (5) for determining a transmission ratio of said coded sound and sai d Icoded 
Pierre ?n accordance w«h the amount of data of at .east one of the said coded sound and sa,d 
roded oicture and outputting the result as an allocation signal, and 

T££^rttm for multiplexing said coded sound and said coded picture based on said 
ailSon Tgna. and further control data including said altocation signal so as to obtam a certam 
transmission frame length and 

: sonars — - — ■ — — ■ ^ piau,e - -* 

control data including an allocation signal from a transmission line 10). 

an audio decoding unit (12) for decoding said coded sound to obtain a decoded 

Tvideo decoding unit (14) for decoding said coded picture to obtain a decoded d,grtal pem 

a decoding control unit (16) for performing control for said audio decoding unrt (12) and said video 

decoding unit (14) based on said allocation signal. 

a D/A converter (13) for converting said decoded digital sound to an audio signal, and 
a D/A converter (15) for converting said decoded digital picture to a video signal. 

. A video and audio multiplex transmission system according to claim 1 wherein *daudk .coding unit 
ffi) is an adaptive type which outputs a plurality of coded sounds with <*^'^™ t ™V£ 
Serene of the said plurality of coded sounds is selected based on an al.ocat.on bit rate output as 

said allocation signal. 

A video and audio multiplex transmission system according to claim 2. wherein said deeqding ^control 
' unrt (1% Performs control based on a demultiplexed allocation signal, which is an ailocation bit rate. 

6 A video and audio multiplex transmission system according to claim 3. wherein said audio coding unit 
f 2 ) Adaptive type which outputs a plurality of coded sounds with different coding bit rates and 
wherein^ne of the said plurality of coded sounds is selected based on an allocation bit rate output as 

~JS£2SE^** 06) performs control based on a demultiplexed allocation signal, i.e.. said 
allocation bit rate. 

7. A video and audio multiplex transmission system according to claim 4. wherein said audio coding unit 
(2) outputs a signal of an optimal audio bit rate. 

8. A video and audio multiplex transmission system according to claim 7. wherein said coding control unit 
(5) outputs the said optimal audio bit rate as it is as said allocation signal. 

9. A video and audio multiplex transmission system according to claim 6. wherein said audio coding unit 
(2) outputs a signal of an optimal bit rate. 

10. A video and audio multiplex transmission system according to claim 9. wherein said coding control unit 
(5) outputs the said optimal audio bit rate as it is as said allocation signal. 

11 A video and audio multiplex transmission system according to claim 1. wherein said audio coding. unit 
(2) hZi SB-ADPCM coding unit which divides said digital sound into a .ow frequency bit porta* .and a 
high frequency bit portion for coding and selects an amount of allocation of the high frequency bit 
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portion by said allocation signaJ. 

12. A video and audio multiplex transmission system according to claim 2, wherein said audio decoding 
unit performs S8-ADPCM decoding of said coded sound based on said allocation signal. 

13. A video and audio multiplex transmission system according to claim 3, wherein said audio coding unit 
(2) is an SB-ADPCM coding unit which divides said digital sound into a low frequency bit portion and a 
high frequency bit portion for coding and selects an amount of allocation of the high frequency bit 
portion by said allocation signal and 

said audio decoding unit performs SB-ADPCM decoding of said coded sound based on said 
allocation signal. 

14. A video and audio multiplex transmission system according to any one of claims 1. 3. 6, and 9 wherein 
said transmission unit is further provided with an interframe rate of change judgement unit (8) which 

is finds a rate of change of interframe data of said digital picture, compares it with a threshold value, and 
outputs results of the judgement 

15. A video and audio multiplex transmission system according to claim 4, wherein said transmission unit is 
further provided with an interframe rate of change judgement unit (8) which finds a rate of change of 
interframe data of said digital picture, compares it with a threshold value, and outputs results of the 
judgement. 
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16. A video and audio multiplex transmission system according to claim 7, wherein said transmission unit is 

further provided with an interframe rate of change judgement unit (8) which finds a rate of change of 

25 interframe data of said digital picture, compares it with a threshold value, and outputs results of the 

judgement 

17. A video and audio multiplex transmission system according to claim 11. wherein said transmission unit 
is further provided with an interframe rate of change judgement unit (8) which finds a rate of change of 

30 interframe data of said digital picture, compares it with a threshold value, and outputs results of the 
judgement 

1a A video and audio multiplex transmission system according to any one of claims 1. 3. 6. or 9. wherein 
said video coding unit (4) is comprised of: 
35 a video coding unit (41) for coding said digital picture. 

a variable length coding unit (42) for coding in variable lengths the said coded video data, 
a buffer (43) for storing temporarily said variable length coded video data, and 

a buffer judgement unit (44) for outputting a storage amount signal in accordance with an amount of 
data stored in said buffer (43) and 
*o outputs said storage amount signal. 

19. A video and audio multiplex transmission system according to claim 4, wherein said video coding unit 
(4) is comprised of: 

a video coding unit (41) for coding said digital picture. 
45 a variable length coding unit (42) for coding in variable lengths the said coded video data, 

a buffer (43) for storing temporarily said variable length coded video data, and 

a buffer judgement unit (44) for outputting a storage amount signal in accordance with an amount of 
data stored in said buffer (43) and 
outputs said storage amount signal. 

so 

20. A video and audio multiplex transmission system according to claim 7, wherein said video coding unit 
(4) is comprised of: 

a video coding unit (41) for coding said digital picture. 

a variable length coding unit (42) for coding in variable lengths the said coded video data. 
55 a buffer (43) for storing temporarily said variable length coded video data, and 

a buffer judgement unit (44) for outputting a storage amount signal in accordance with an amount of 
data stored in said buffer (43) and 
outputs said storage amount signal. 
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21. A video and audio multiplex transmission system according to claim 11, wherein said video coding unit 
(4) is comprised of: 

a video coding unit (41) for coding said digital picture. 

a variable length coding unit (42) for coding in variable lengths the said coded video data, 

a buffer (43) for storing temporarily said variable length coded video data, and 

a buffer judgement unit (44) for outputting a storage amount signal in accordance with an amount of 

data stored in said buffer (43) and 

outputs said storage amount signal. 

22. A video and audio multiplex transmission system according to claim 16, wherein said video coding unit 
(4) is comprised of: 

a video coding unit (41) for coding said digitai picture. 

a variable length coding unit (42) for coding in variable lengths the said coded video data, 

a buffer (43) for storing temporarily said variable length coded video data, and 

a buffer judgement unit (44) for outputting a storage amount signal in accordance w.th an amount of 

data stored in said buffer (43) and 

outputs said storage amount signal. 

23. A video and audio multiplex transmission system according to claim 17. wherein said video coding unit 
(4) is comprised of: 

a video coding unit (41) for coding said digital picture, 

a variable length coding unit (42) for coding in variable lengths the said coded video data, 

a buffer (43) for storing temporarily said variable length coded video data, and 

a buffer judgement unit (44) for outputting a storage amount signal in accordance with an amount of 

data stored in said buffer (43) and 

outputs said storage amount signal. 

24. A video and audio multiplex transmission system according to any one of claims 15 and 17. wherein 
said coding control unit (5) generates an allocation signal based on the results of judgement output by 
said interframe rate of change judgement unit (8). 

25. A video and audio multiplex transmission system according to any one of claims 19 and 21. wherein 
said coding control unit (5) generates an allocation signal based on the storage amount signal. 

26. A video and audio multiplex transmission system according to claim 16. wherein said coding control 
unit (5) generates an allocation signal in accordance with the results of judgement of the interframe rate 
of change and an optimal audio bit rate. 

27 A video and audio multiplex transmission system according to any one of claims 20 and 22. wherein 
said coding control unit (5) generates an allocation signal in accordance with the storage amount signal 
and optimal audio bit rate. 

2a A video and audio multiplex transmission system according to claim 22, wherein said coding control 
unit (5) generates an allocation signal in accordance with the results of judgement of the interframe rate 
of change, storage amount signal, and optimal audio bit rate. 

29 A video and audio multiplex transmission system according to claim 23. wherein said coding control 
unit (5) generates an allocation signal in accordance with the results of judgement of the interframe rate 
of change and the storage amount signal. 

30. A video and audio multiplex transmission system according to any one of claims 1. 3, 4. 6. 7. 8. 9.10. 
11 13 15 16. 17, 19, 20, 21. 22, and 23. wherein said transmission unit (B) is further provided witn a 
delay mount calculation unit (31) which generates delay time data for synchronizing video reproduced 
output and audio reproduced output from input and output data of said video coding unit (4) and saio 
delay time data is multiplexed together by the multiplexing unit (6). 

31 A video and audio multiplex transmission system according to any one of claims 2. 3. 5. and 12. 
" wherein said reception unit (B) is further provided with a variable delay control unit (32) for delaying the 
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said audio data in accordance with said delay time data. 



32. A video ^and audio multiplex transmission system having a transmission unit (D) comprising: 

an A/D converter (1 ) for converting audio input to digital sound, 
s an audio coding unit (2) for coding said digital sound. 

an A/D converter (3) for converting video input to a digital picture 

rJr^L^u^TT*? ^ j 31> f ° r 9enerati " 9 d9,ay time data for synchronizing video 
reproduced output and audio reproduced output from input and output data of said video coding unit 

io ^multiplexing unit (6) for multiplexing said coded video data and audio data and said delay time 

33. A video and audio multiplex transmission system having a reception unit (E) comprising: 

Ldf Q m dT Tl (1 demult, 'P ,exin 9 * multiplexed signal from a transmission line (10) into 
is audio data, video data, and delay time data, 

ajariable delay control unit (32) for delaying said audio data in accordance with said delay time 

an audio decoding unit (12) for decoding audio data from said variable delay control unit (32) 
a D/A converter (13) for converting said decoded digital sound to an audio signal 
20 a video decoding unit (14) for decoding said coded picture, and ' 

a D/A converter (15) for converting said decoded digital picture to a video signal. 

34. A video and audio multiplex transmission system having a transmission unit (D) comprising: 

an A/D converter (1) for converting audio input to digital sound. 
25 an audio coding unit (2) for coding said digital sound. 

an A/D converter (3) for converting video input to a digital picture 

Lrod^T^\ Ca H U,a ^ n ^ (31> f ° r 9 eneratln 9 d9,a y *™ ^ta for synchronizing video 
(4) a^d reproduced output from input and output data of said video coding unit 

30 data U and eXin9 ^ **" mU,tip ' exin9 Said Coded v,deo data and audi ° d *a and said delay time 

a Teception unit (E) comprising: 

L2fo m H U iL P,eX H 9 lU) J° r demu,ti P ,8xin S a multiplexed signal from a transmission line (10) into 
audio data, video data, and delay time data. 

35 datT^' 9 d6lay C ° ntr< " U " rt P2) f ° r d9,ayin9 S3id 3Udl ° data in acc ° r °"ance with said delay time 

an audio decoding unit (12) for decoding audio data from said variable delay control unit (32) 
a D/A converter (1 3) for converting said decoded digital sound to an audio signal 
a video decoding unit (14) for decoding said coded picture, and 

« a D/A converter (1 5) for converting said decoded digital picture to a video signal. 

35. A video and audio multiplex transmission system which is provided, to deal with video signals with a 

i is provided, to deal with audio signals, with an audio signal coding unit (2). and 

is comprised so that the output from said variable length coding unit (42) and the output 

2 t0 S3id aUd '° C ° dinS UOit (2> are multiplexed and transmitted, the receiver side 
extract the v.deo signal and audio signal from what is transmitted 

said video and audio multiplex transmission system being provided with a time division coding unit 
(71) for extracting a signal during an effective period of a sound and forming it into a packet based 
on output from said audio coding unit (2), 

said time division coding unit (71) is comprised so as to notify a system control unit (19) of an audio 
transmission speed. 

nn^TrZSTf. T (19) iS com P risad so a * to receive said audio transmission speed and 
modify threshold data for controlling the coding amount of said video coding unit (41 ) corresponding 
to the amount of data m a buffer memory (43) in said variable length coding unit (42). and 
performing transmission in accordance with a frame format suited for sa,d audio transm.ss.on speed. 
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